Human Activity Recognition from Video: modeling, feature selection and classification architecture

In this paper, we address the problem of recognizing human activities, such as {Active, Inactive, Walking, Running, Fighting} from video sequences, with a particular emphasis on the problems of feature selection, data modeling and classifier structure. The need for such systems is increasing everyday, with the number of (hundreds or thousands) of surveillance cameras deployed in public spaces. This massive number of cameras calls for systems able to detect, categorize and recognize human activity, requesting human attention only when necessary. Our work is focused on three fundamental issues: (i) the design of a classifier and data modeling for activity recognition; (ii) how to perform feature selection and (iii) how to define the structure of a classifier. We use of a Bayesian classifier, and model the likelihood functions as Gaussian mixtures, adequate to cope with complex data distributions, that are learned automatically. As for feature selection, we propose several (suboptimal) methods to evaluate the recognition rate achieved with different feature combinations, with the Bayesian classifier. Finally, we investigate the use of hierarchical classifiers (including the possibility of automatic generation). Our results were based on nearly 16,000 images of five activities and we achieved an error rate as low as 1.5%. These experiments clearly demonstrate the importance of powerful methodologies for data modeling and how intertwined feature selection, classifier design and the structure of the classifier are.